Recently, the performance and functionality of digital cameras and digital movie cameras that use some solid-state image sensor such as a CCD and a CMOS (which will be sometimes simply referred to herein as an “image sensor”) have been enhanced to an astonishing degree. In particular, the size of a pixel structure for use in a solid-state image sensor has been further reduced these days thanks to rapid development of semiconductor device processing technologies, thus getting an even greater number of pixels and drivers integrated together in a solid-state image sensor. As a result, the resolution of an image sensor has lately increased rapidly from around one million pixels to ten million or more pixels in a matter of few years. On top of that, the quality of an image captured has also been improved significantly as well.
As for display devices, on the other hand, LCD and plasma displays with a reduced depth now provide high-resolution and high-contrast images, thus realizing high performance without taking up too much space. And such video quality improvement trends are now spreading from 2D images to 3D images. In fact, 3D display devices that achieve high image quality although they require the viewer to wear a pair of polarization glasses have been developed just recently.
As for the 3D image capturing technology, a typical 3D image capture device with a simple arrangement uses an image capturing system with two cameras to capture a right-eye image and a left-eye image. According to the so-called “two-lens image capturing” technique, however, two cameras need to be used, thus increasing not only the overall size of the image capture device but also the manufacturing cost as well. To overcome such a problem, methods for capturing multiple images with parallax (which will be sometimes referred to herein as a “multi-viewpoint image”) by using a single camera have been researched and developed. Such a method is called a “single-lens image capturing method”.
For example, Patent Document No. 1 discloses a scheme that uses two polarizers, of which the transmission axes cross each other at right angles, and a rotating polarization filter. FIG. 11 is a schematic representation illustrating an arrangement for an image capturing system that adopts such a scheme. This imaging area includes a 0-degree-polarization polarizer 11, a 90-degree-polarization polarizer 12, a reflective mirror 13, a half mirror 14, a circular polarization filter 15, a driver 16 that rotates the circular polarization filter 15, an optical lens 3, and an image capture device 29 which captures the image that has been produced by the optical lens 3. In this arrangement, the half mirror 14 reflects the light that has been transmitted through the polarizer 11 and then reflected from the reflective mirror 13 but transmits the light that has been transmitted through the polarizer 12. With such an arrangement, the light beams that have been transmitted through the two polarizers 11 and 12, which are arranged at two different positions, pass through the half mirror 14, the circular polarization filter 15 and the optical lens 3 and then enter the image capture device 29, where an image is captured. The image capturing principle of this scheme is that two images with parallax are captured by rotating the circular polarization filter 15 so that the light beams that have been incident on the two polarizers 11 and 12 are imaged at mutually different times.
According to such a scheme, however, images at mutually different positions are captured time-sequentially by rotating the circular polarization filter 15, and therefore, two images with parallax cannot be captured at the same time, which is a problem. On top of that, since all of the incoming light passes through the polarizers 11, 12 and the polarization filter 15, the quantity of the light received eventually by the image capture device 29 decreases by as much as 50%, which is non-negligible, either.
To overcome these problems, Patent Document No. 2 discloses a scheme for capturing two images with parallax at the same time without using such mechanical driving. An image capture device that adopts such a scheme gets the two incoming light beams, which have come from two different directions, condensed by a reflective mirror, and then received by an image sensor in which two different kinds of polarization filters are arranged alternately, thereby capturing two images with parallax without using mechanical driving.
FIG. 12 is a schematic representation illustrating an arrangement for an image capturing system that adopts such a scheme. This image capturing system includes two polarizers 11 and 12, of which the transmission axes are arranged to cross each other at right angles, reflective mirrors 13, an optical lens 3, and an image sensor 2. On its imaging area, the image sensor 2 has a number of pixels 10 and polarization filters 17 and 18, each of which is provided one to one for an associated one of the pixels. Those polarization filters 17 and 18 are arranged alternately over all of those pixels. In this case, the transmission axis directions of the polarization filters 17 and 18 agree with those of the polarizers 11 and 12, respectively.
With such an arrangement, the incoming light beams are transmitted through the polarizers 11 and 12, reflected from the reflective mirrors 13, passed through the optical lens 3 and then incident on the imaging area of the image sensor 1. Those light beams to be transmitted through the polarizers 11 and 12, respectively, and then incident on the image sensor 1 are transmitted through the polarization filters 17 and 18 and then photoelectrically converted by the pixels that are located right under those polarization filters 17 and 18. If the images to be produced by those light beams that have been transmitted through the polarizers 11 and 12 and then incident on the image sensor 1 are called a “right-eye image” and a “left-eye image”, respectively, then the right-eye image and the left-eye images are generated by a group of pixels that face the polarization filters 17 and a group of pixels that face the polarization filter 18, respectively.
As can be seen, according to the scheme disclosed in Patent Document No. 2, two kinds of polarization filters, of which the transmission axes are arranged so as to cross each other at right angles, are arranged alternately over the pixels of the image sensor, instead of using the circular polarization filter disclosed in Patent Document No. 1. As a result, although the resolution decreases to a half compared to the method of Patent Document No. 1, a right-eye image and a left-eye image with parallax can be obtained at the same time by using a single image sensor. According to such a technique, however, the incoming light has its quantity decreased considerably when being transmitted through the polarizers and the polarization filters, and therefore, the quantity of the light received by the image sensor decreases as significantly as in Patent Document No. 1.
To cope with such a problem of the decreased quantity of light received, Patent Document No. 3 discloses a technique for obtaining two images with parallax and a normal image with a single image sensor. According to such a technique, those two images with parallax and the normal image can be obtained by a single image sensor by changing mechanically some components that have been used to capture two images with parallax with alternative components for use to capture a normal image, and vice versa. When two images with parallax are going to be obtained, two polarization filters are arranged on the optical path as disclosed in Patent Document No. 2. On the other hand, when a normal image is going to be obtained, those polarization filters are mechanically removed from the optical path. By introducing such a mechanism, those images with parallax and a normal image that uses the incoming light highly efficiently can be obtained.
Although a polarizer or a polarization filter is used according to the techniques disclosed in Patent Document Nos. 1 to 3, color filters may also be used according to another approach. For example, Patent Document No. 4 discloses a technique for obtaining two images with parallax at the same time using color filters. FIG. 13 schematically illustrates an image capturing system as disclosed in Patent Document No. 4. The image capturing system includes a lens 3, a lens diaphragm 19, a light beam confining plate 20 with two color filters 20a and 20b that have mutually different transmission wavelength ranges, and a photosensitive film 21. In this case, the color filters 20a and 20b may be filters that transmit red- and blue-based light rays, respectively.
In such an arrangement, the incoming light passes through the lens 3, the lens diaphragm 19 and the light beam confining plate 20 and produces an image on the photosensitive film. In the meantime, only red- and blue-based light rays are respectively transmitted through the two color filters 20a and 20b of the light beam confining plate 20. As a result, a magenta-based color image is produced on the photosensitive film by the light rays that have been transmitted through the two color filters. In this case, since the color filters 20a and 20b are arranged at mutually different positions, the image produced on the photosensitive film comes to have parallax. Thus, if a photograph is developed with the photosensitive film and viewed with a pair of glasses, in which red and blue films are attached to its right- and left-eye lenses, the viewer can view an image with depth. In this manner, according to the technique disclosed in Patent Document No. 4, a multi-viewpoint image can be produced using the two color filters.
According to the technique disclosed in Patent Document No. 4, the light rays are imaged on the photosensitive film, thereby producing images with parallax there. Meanwhile, Patent Document No. 5 discloses a technique for producing images with parallax by transforming incoming light into electrical signals. FIG. 14 schematically illustrates a light beam confining plate according to Patent Document No. 5. According to such a technique, a light beam confining plate 22, which has a red ray transmitting R area 22R, a green ray transmitting G area 22G and a blue ray transmitting B area 22B, is arranged on a plane that intersects with the optical axis of the imaging optical system at right angles. And by getting the light rays that have been transmitted through those areas received by a color image sensor that has red-, green- and blue-ray-receiving R, G and B pixels, an image is generated based on the light rays that have been transmitted through those areas.
Patent Document No. 6 also discloses a technique for obtaining images with parallax using a similar configuration to the one used in Patent Document No. 5. FIG. 15 schematically illustrates a light beam confining plate as disclosed in Patent Document No. 6. According to that technique, by making the incoming light pass through R, G and B areas 23R, 23G and 23B of the light beam confining plate 23, images with parallax can also be produced.
Patent Document No. 7 also discloses a technique for generating multiple images with parallax using a pair of filters with mutually different colors, which are arranged symmetrically to each other with respect to an optical axis. By using red and blue filters as the pair of filters, an R pixel that senses a red ray observes the light that has been transmitted through the red filter, while a B pixel that senses a blue ray observes the light that has been transmitted through the blue filter. Since the red and blue filters are arranged at two different positions, the light received by the R pixel and the light received by the B pixel have come from mutually different directions. Consequently, the image observed by the R pixel and the image observed by the B pixel are ones viewed from two different viewpoints. And by defining corresponding points between those images on a pixel-by-pixel basis, the magnitude of parallax can be calculated.
And based on the magnitude of parallax calculated and information about the focal length of the camera, the distance from the camera to the subject can be obtained. Patent Document No. 8 discloses a technique for obtaining information about a subject distance based on two images that have been generated using either a diaphragm to which two color filters with mutually different aperture sizes are attached or a diaphragm to which two color filters in two different colors are attached horizontally symmetrically with respect to the optical axis. According to such a technique, if light rays that have been transmitted through red and blue color filters with mutually different aperture sizes are observed, the degrees of blur observed vary from one color to another. That is why the degrees of blur of the two images that are associated with the red and blue color filters vary according to the subject distance. By defining corresponding points with respect to those images and comparing their degrees of blur to each other, information about the distance from the camera to the subject can be obtained. On the other hand, if light rays that have been transmitted through two color filters in two different colors that are attached horizontally symmetrically with respect to the optical axis are observed, the direction from which the light observed has come changes from one color to another. As a result, two images that are associated with the red and blue color filters become images with parallax. And by defining corresponding points with respect to those images and calculating the distance between those corresponding points, information about the distance from the camera to the subject can be obtained.
According to the techniques disclosed in Patent Documents Nos. 4 to 8 mentioned above, images with parallax can be produced by arranging RGB color filters on a light beam confining plate. However, since a light beam confining plate is used, the percentage of the incoming light that can be used decreases significantly. In addition, to increase the effect of parallax, those RGB color filters should be arranged at distant positions and should have decreased areas. In that case, however, the percentage of the incoming light that can be used further decreases.
Unlike these techniques, Patent Document No. 9 discloses a technique for obtaining multiple images with parallax and a normal image that is free from the light quantity problem by using a diaphragm in which RGB color filters are arranged. According to that technique, when the diaphragm is closed, only the light rays that have been transmitted through the RGB color filters are received. On the other hand, when the diaphragm is opened, the RGB color filter areas are outside of the optical path, and therefore, the incoming light can be received entirely. Consequently, images with parallax can be obtained when the diaphragm is closed and a normal image that uses the incoming light highly efficiently can be obtained when the diaphragm is opened.